Direction-Aware Semi-Dense SLAM

To aide simultaneous localization and mapping (SLAM), future perception systems will incorporate forms of scene understanding. In a step towards fully integrated probabilistic geometric scene understanding, localization and mapping we propose the first direction-aware semi-dense SLAM system. It jointly infers the directional Stata Center World (SCW) segmentation and a surfel-based semi-dense map while performing real-time camera tracking. The joint SCW map model connects a scene-wide Bayesian nonparametric Dirichlet Process von-Mises-Fisher mixture model (DP-vMF) prior on surfel orientations with the local surfel locations via a conditional random field (CRF). Camera tracking leverages the SCW segmentation to improve efficiency via guided observation selection. Results demonstrate improved SLAM accuracy and tracking efficiency at state of the art performance. Future perception systems in applications such as autonomous cars, autonomous robots, or augmented reality will integrate scene understanding into the purely geometric localization and mapping task. This is likely to improve both simultaneous localization and mapping (SLAM), provide a basis for higher-level reasoning about the scene, and richer information for human operators. In current systems scene understanding is used in two ways: (1) to improve the operation of the 3D perception system and (2) to provide additional information for higher-level inference or a human operator. We argue that only systems in the first class actually “understand” aspects of the scene because they are able to use inferred concepts to improve on their other inferential tasks (i.e. localization and mapping). The number of systems that fall into this class is still small. In order to improve 3D reconstruction and localization via scene understanding most approaches rely on geometric scene priors such as planarity [10, 39, 20, 28], the Manhattan World [37] (MW) or the Stata Center World [6] (SCW) assumption. The assumption of planarity is a local assumption that cannot explain the connection between disparate scene parts like all the parallel planes in typical man-made environments. Such connections can be captured and explained by global assumptions such as the MW and the SCW assumption. Because the MW assumption is limited Figure 1: We propose the first direction-aware semi-dense SLAM system. Based on the Stata Center World assumption the system jointly infers a directional segmentation (right) and a semi-dense surfel-based map (left) in real-time. to very specific environments, we instead explore the flexible Stata Center World model to improve 3D reconstruction and camera tracking. As shown in [43, 44], the directional clustering of a scene’s surface normals under the Stata Center World implies a segmentation that captures scene-wide regularities of the environment [45] as can be seen in the segmentation in Fig. 1. Based on the SCW scene prior, we propose the first semi-dense nonparametric direction-aware SLAM system. It performs joint inference over the Bayesian nonparametric SCW scene segmentation and the world map using Gibbssampling without precluding real-time operation. To connect the scene-wide Stata Center World model with local surface properties we model the assumption that nearby areas in the same directional segment are likely planar using a CRF. We demonstrate experimentally that using the directional segmentation improves SLAM accuracy and camera tracking efficiency via guided observation selection.